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(57) Abstract 

A speech recognition apparatus for use in an environment containing ambient acoustic noise includes a speech recogniser (20) for 
receiving and processing data representative of a speech utterance to be recognised to output data representative of or dependent on the 
lexical content of said utterance. The apparatus includes level adjusting means (16) for adjusting the level of the speech utterance. The 
apparatus also includes means for measuring said ambient acoustic noise level before or during said utterance, processing means (22) for 
using said measured acoustic noise level to predict the likely level of the speech utterance, and for adjusting the gain of the signal in 
accordance with said prediction of the likely level of the speech utterance. 
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Speech Processing 



This invention relates to apparatus and a method for 
estimating the speech level of a speaker exposed to an 
environment containing a variable amount of acoustic noise. 
5 In particular, but not exclusively, the invention 

relates to such apparatus and methods for use in speech 
recognition . 

The central process in automatic speech recognition is 
the comparison between some representation of the speech to 

10 be recognised and a set of reference models corresponding to 
speech sounds or words or other units. It is important that 
the level of the speech signal represented ,in the recogniser 
should be close to that expected by the models. 

Because speech sounds vary in their intrinsic loudness, 

15 measuring overall speech level is not a trivial process. It 
is necessary either to take a large enough sample of the 
speech that the variations occurring between speech sounds 
average out, or to compare an utterance whose level is to be 
measured with an utterance at some known level whose 

20 phonetic content is the same. In this second method, 
phonetically identical speech sounds can be compared, but it 
does require a knowledge of the content of the utterance to 
be measured. 

We have realised that it is in fact possible to 
25 estimate variations in the likely level of the speech signal 
in acoustically noisy environments by measuring the ambient 
noise level and using a phenomenon known as the Lombard 
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Effect to determine the likely speech levels. The Lombard 
Effect is the phenomenon that when people are exposed to 
noise their speech changes and becomes generally becomes 
louder. If no adjustment is made for the Lombard Effect in 
5 an automatic speech recognition system there will be a 
mismatch between the level of the speech to be recognised 
and the expected level. In principle, this could be 
corrected by observing the speech level and adjusting the 
gain of an amplifier in the recogniser to compensate for the 

10 variation in level. However, in some circumstances this is 
not a practical arrangement. For example, in a car the 
noise level can change from one utterance to another 
following changes in the speed of the car or in the road 
surface, or because a window is wound down. A gain setting 

15 based on the previous utterance will then be inappropriate. 
In some circumstances, it might be possible to wait until 
the utterance was complete, measure the speaking level, 
adjust the recorded utterance to normalise this level, and 
only then submit it to the recogniser. However, this 

20 process would introduce a delay in the response of the 
recogniser, which for many applications would be 
unacceptable . 

In one aspect, this invention provides apparatus for 
predicting the speech level of a speaker exposed to an 
25 environment containing a variable level of ambient acoustic 
noise, the apparatus comprising means for measuring said 
ambient acoustic noise level, and processing means for using 
said measured acoustic noise level to predict the likely 



WO 00/23984 




PCT/GB99/03322 



3 



speech level . 

In this apparatus, as the noise level in the 
environment in which the speaker is located changes between 
utterances, so his speech level is likely to rise and fall 
5 in accordance with the Lombard Effect, and the apparatus 
predicts the likely speech level. We have found that the 
likely speech level can be predicted with reasonable 
accuracy by measuring the noise immediately adjacent to an 
utterance; measuring the level of a steady noise is quite 

10 simple and can be carried out with just a short sample of 
the noise. The apparatus preferably also uses a measure of 
the speech level and the corresponding noise level relating 
to a previous or standardised utterance. 

The ambient acoustic noise level could be measured 

15 before, after or even during utterance of a word or phrase, 
and it is preferred for the measurement to be made close in 
time to the utterance to reduce the possibility of the 
prediction of the likely speech level being inaccurate due 
to a significant shift in noise level between measurement 

20 and the actual utterance. 

It is preferred for the measuring means to measure the 
ambient acoustic noise level immediately before the 
utterance, the estimate of speech level being determined 
before or as the utterance is made rather than thereafter. 

25 Alternatively the measurement may be after the utterance. 

The apparatus preferably includes means operable to 
define, for each utterance, an utterance period comprising 
a first time period for measuring said acoustic noise level 
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and a second time period during which said utterance is 
made . 

Thus in a preferred embodiment, the apparatus includes 
a user input device (such as e.g. a switch) and a timer and 
5 control means for defining said first noise measuring 
period, and said second speech measuring and/or recording 
period, the end of said first period being indicated to said 
user. 

In a particularly preferred aspect, said apparatus is 

10 responsive to a succession of one or more utterances by a 
speaker and said measuring means measures the ambient noise 
level prevailing at each of said utterances to provide a 
series of noise measurements and said apparatus includes 
means for measuring the speech level of an utterance, and 

15 said processing means uses at least two of said noise 
measurements, together with the measurement of the speech 
level of the immediately previous utterance, to produce the 
prediction of the speech level of the most recent utterance. 
In one example, where the noise is measured immediately 

2 0 before an utterance, the processing apparatus means predicts 
the speech level S x * of an utterance (1) on the basis of the 
following expression: 

S x * = S 0 +f (No-Nj 
where S Q is the speech level of the immediately previous 

25 utterance; N X ,N 0 are the noise levels prevailing immediately 
before the utterance whose speech level is to be estimated, 
and immediately before the next previous utterance 
respectively, and f (x) is a function relating changes in the 
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noise level in which the speaker is situated to the 
speaker's speech level. 

The function is preferably monotonic increasing, and in 
a simple case is a multiplying factor less than 1. The 
5 multiplying factor may typically be a positive value in the 
range of from 0 to 0.6, and in one example is 0.32. 

Alternatively the function may be a more complex 
function of the noise level difference. Likewise, the 
function may be modified to take account of more than just 
10 two noise level measurements; thus information relating to 
the speech levels of several previous utterances, together 
with the associated noise levels may be aggregated to 
predict the speech level of the next utterance. 

In another aspect, this invention provides speech 
15 recognition or processing apparatus including predicting 
apparatus as set out above for use in adjusting the gain of 
the speech signal prior to recognition processing. 

In yet another aspect, this invention provides a method 
for predicting the speech level of a speaker exposed to an 
20 environment containing a variable level of ambient acoustic 
noise, said method comprising the steps of : - 

measuring said ambient acoustic noise level , and 
processing said measured acoustic noise level to 
produce a prediction of the likely speech level. 
25 In a further aspect, this invention provides a method 

for controlling the gain in a speech recognition or 
processing system, which comprises controlling the gain of 
the speech signal in accordance with a prediction of the 
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speech level obtained by the above method. 

Whilst the invention has been described above, it 
extends to any inventive combination of the features set out 
above or in the following descriptions. 
5 The invention may be performed in various ways, and an 

embodiment thereof will now be described by way of example 
only, reference being made to the accompanying drawing in 
which : - 

Figure 1 is a block diagram of a speech recogniser 
10 incorporating speech level prediction in accordance with the 
invention. 

The illustrated embodiment implements a system which 
applies knowledge of variation in the ambient acoustic noise 
level and its likely effect on the speech level to predict 

15 the speech level in the next utterance to be recognised by 
a speech recogniser. It is assumed that the variation in 
noise level over the duration of a single utterance is small 
compared with the variations occurring between utterances, 
and also that the noise has sufficient short-term 

2 0 stationarity that its level can be measured from a brief 
sample . 

Referring to Figure 1, the speech recognition system 
comprises a microphone 10 whose output is subjected to voice 
processing at 12 before analogue to digital conversion at 
25 14. The digital signal passes via a digital gain device 16 
to a processor 18 which incorporates a recogniser 2 0 and a 
speech level estimator 22 . The speech recogniser may be of 
any suitable type and examples of suitable recognisers will 
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be well known to those skilled in the art. The processor 18 
also receives an input from a switch 24 acting as a user 
input device, and can issue warning tones to the user 
through a sounder 26. 
5 The system illustrated is intended for use in a noisy 

environment whose noise level varies. In use, the user 
alerts the system when he wants to make an utterance to be 
recognised, by closing the switch 24. The processor then 
defines an utterance frame, comprising a first short time 
10 period, during which the ambient noise is sampled, followed 
by issuing a tone on the sounder 26, which indicates to the 
user that he may speak, followed by a second period during 
which the speech signal is sampled and sent to the 
recogniser 20. The second period is longer than the first 
15 period and sufficiently long to contain the longest 
utterance to be recognised. There are a number of ways of 
delimiting the second period other than providing a period 
of set duration. For example the length of the period may 
be user designated, e.g. by the user keeping the button 
20 pressed or pressing the button again. Alternatively, the 
processor may listen for a period of silence, or it may 
infer the end of a command based on an analysis of the 
grammar of the utterance. In addition, instead of using a 
switch, the start of the utterance frame may be marked by 
25 the user uttering a codeword. 

Since it is known that speech levels vary with noise 
level, it is possible to predict a change in the speech 
level in an utterance from a change in the noise level . The 
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speech and noise levels, S 0 and N 0/ (in dB units) are measured 
by the processor in one noise condition. The new noise 
level, N 1# in the first period of the next utterance, just 
before the start of an utterance to be recognised, is also 
5 measured by the processor. The difference in the two noise 
levels, Nq-Nj, is then determined and used by the processor, 
together with knowledge of the speech level, S G of the 
previous utterance, to predict the speech level, S 1# of the 
new utterance. We can write S\ = S 0 + jf (N,,-!^) , where S\ is 

10 a prediction estimate of S x and f{x) is the function relating 
changes in the noise level in the speaker' s ears to the 
speaker's speech level. In the simplest arrangement, the 
function is a multiplying factor less than 1, but it can 
also be a more complex function of the noise level 

15 difference. In practice we have determined empirically that 
the speech level good results are achieved in one 
application by using a multiplying factor of typically 0.3 
although positive values between 0 and 0.6 should all 
provide some improvement. It may be assumed to be the same 

2 0 for all speakers or may be estimated separately for each 
speaker . 

Since the measurements of the reference speech and 
noise levels, S 0 and N 0 , respectively, are subject to 
measurement errors, it may be preferred to aggregate the 
25 information contributing to the prediction of S x from several 
previous utterances and noise estimates. The computation of 
S*j described in the previous paragraph can be replaced by an 
average over several previous utterances. This may be a 



WO 00/23984 




PCT/GB99/03322 



simple average or it may be a weighted average, the weights 
possibly depending on factors such as the time difference 
between the various reference utterances and S 1 and on the 
relative durations of the various reference utterances. For 
5 example the computation may take account of any time 
effects. For example it may be found that, when exposed to 
a particular level of ambient noise that the speaker's 
speech level rises over an initial period and then 
decreases, in a temporal filtering effect. 

10 Having determined an estimate of the speech level of 

the new utterance, the processor controls the gain of the 
signal accordingly. The gain may be adjusted at various 
points; it may be adjusted whilst the signal is still in the 
analogue domain or it may be achieved by digital scaling as 

15 shown by the digital gain device 16. A further alternative 
is to manipulate the fast fourier transform (FFT) values in 
the speech recogniser. If a cepstrum is computed, the 
signal may be scaled by adding an appropriate constant to 
the C Q coefficient. In a further arrangement, the system may 

2 0 compensate for increases or decreases in the speech level by 
adjusting the effective speech levels that the models in the 
recogniser represent . 

The gain may take into account factors other than 
simply the level of the background noise; for example it 

25 could also take account of its spectral structure. 

The output of the recogniser may be used in any 
convenient form. For example it could be used to enable a 
person to issue spoken commands to equipment . 
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Claims 

1. Apparatus for predicting the speech level in an 
utterance of a speaker exposed to an environment containing 
a variable level of ambient acoustic noise, the apparatus 

5 comprising means for measuring said ambient acoustic noise 
level, and processing means for using said measured acoustic 
noise level to predict the likely speech level in said 
utterance . 

2. Apparatus according to Claim 1, wherein said measuring 
10 means measures the ambient acoustic noise level immediately 

adjacent to said utterance. 

3. Apparatus according to Claim 2, including means for 
activating said measuring means before the utterance. 

4 . Apparatus according to an preceding Claim which 
15 includes means operable to define, for each utterance, an 

utterance period comprising a first time period for 
measuring said acoustic noise level and a second time period 
during which said utterance is made. 

5. Apparatus according to Claim 4, which includes a user 
20 input device, a timer, control means for defining said first 

period, and said second period, and means for indicating to 
a user the end of said first period. 

6. Apparatus according to Claim 5, wherein said apparatus 
is responsive to a succession of one or more utterances by 

25 a speaker, and said measuring means is operable to measure 
the ambient noise level prevailing at each of said 
utterances to provide a series of noise values, and said 
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apparatus includes means for measuring the speech level of 
an utterance, and said processing means uses at least two of 
said noise" values, together with a value representative of 
the speech level of the immediately previous utterance, to 
5 predict the likely speech level of the next utterance. 

7. Apparatus according to Claim 6, wherein said measuring 
means is .adapted to measure the ambient acoustic noise level 
before an utterance, and the processing means estimates the 
speech level S x * of an utterance (1) on the basis of the 
10 following expression: 

S/ = S 0 +f (N 0 -N x ) 

where 

S Q is the speech level of the immediately previous 
utterance ; 

15 N lf N 0 are the noise levels prevailing immediately before 

the utterance whose speech level is to be estimated, and 
immediately before the next previous utterance respectively, 
and 

f (x) is a function relating changes in the noise level 
20 in which the speaker is situated to the speaker's speech 
level . 

8. Apparatus according to Claim 7, wherein said processing 
means predicts the speech level S y * on the basis of the 
following expression: 

where k is a constant, k>l. 

9. Apparatus according to Claim 8, wherein k lies in the 
range of from 0 to 0.6. 
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10. A speech recognition apparatus for use in an 
environment containing ambient acoustic noise, said 
apparatus including speech recogniser means for receiving 
and processing data representative of a speech utterance to 
5 be recognised to output data representative of or dependent 
on the lexical content of said utterance, said apparatus 
including level adjusting means for adjusting the level of 
the speech utterance, said apparatus further including means 
for measuring said ambient acoustic noise level before or 

10 during said utterance, processing means for using said 
measured acoustic noise level to predict the likely level of 
the speech utterance, and means for adjusting said level 
controlling means in accordance with said prediction of the 
likely level of the speech utterance. 

15 11. A method for predicting the speech level of an 
utterance of a speaker exposed to an environment containing 
a variable level of ambient acoustic noise, said method 
comprising the steps of : - 

measuring said ambient acoustic noise level, and 

2 0 processing said measured acoustic noise level to 

predict the likely speech level of said utterance. 
12 A method according to Claim 11, wherein said ambient 
acoustic noise level is measured before said utterance. 
13. A method according to Claim 11, wherein a plurality of 

25 measurements of said acoustic noise level is taken and used 
with one or measurements of the speech levels corresponding 
to said measurements of acoustic noise level to predict the 
likely speech level of the utterance. 
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14. A method for controlling the gain in a speech 
recognition or processing system in an environment 
containing a variable level of ambient acoustic noise, which 
method comprises controlling the gain of the speech signal 
5 in accordance with an estimate of the speech level, said 
estimate being obtained by measuring said ambient acoustic 
noise level, and processing said measured acoustic noise 
level to produce an estimate of the likely speech level of 
said utterance. 
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ABSTRACT 



A speech recognition apparatus for use in an environment containing ambient acoustic 
noise includes a speech recognizer (20) for receiving and processing data representative of a 
speech utterance to be recognized to output data representative of or dependent on the lexical 
content of the utterance. The apparatus includes level adjusting elements (16) for adjusting 
the level of the speech utterance. The apparatus also includes elements for measuring the said 
ambient acoustic noise level before or during the utterance, processing elements (22) for 
using the measured acoustic noise level to predict the likely level of the speech utterance, and 
for adjusting the gain of the signal in accordance with the prediction of the likely level of the 
speech utterance. 
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I. Basis of the report 
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response to an invitation under Articie 14 are referred to in this report as "originally fiied" and are not annexed to 
the report since they do not contain amendments (Rules 70. 16 and 70. 1 7).): 

Description, pages: 

1-9 as originally filed 

Claims, No.: 

1-14 as originally filed 

Drawings, sheets: 

1/1 as originally filed 

2. With regard to the language, all the elements marked above were available or furnished to this Authority in the 
language in which the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language: , which is: 

□ the language of a translation furnished for the purposes of the international search (under Rule 23.1 (b)). 

□ the language of publication of the international application (under Rule 48.3(b)). 

□ the language of a translation furnished for the purposes of international preliminary examination (under Rule 
55.2 and/or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the 
international preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in written form. 

□ filed together with the international application in computer readable form. 

□ furnished subsequently to this Authority in written form. 

□ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in 
the international application as filed has been furnished. 

□ The statement that the information recorded in computer readable form is identical to the written sequence 
listing has been furnished. 

4. The amendments have resulted in the cancellation of: 

□ the description, pages: 

□ the claims, Nos.: 
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□ the drawings, sheets: 

5. □ This report has been established as if (some of) the amendments had not been made, since they have been 

considered to go beyond the disclosure as filed (Rule 70.2(c)): 

(Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this 
report.) 

6. Additional observations, if necessary: 



V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 

1. Statement 



Novelty (N) 


Yes: 


Claims 


6-9, 13 




No: 


Claims 


1-5, 10-12, 14 


Inventive step (IS) 


Yes: 


Claims 


6-9, 13 




No: 


Claims 


1-5, 10-12, 14 


Industrial applicability (IA) 


Yes: 


Claims 


1-14 




No: 


Claims 





2. Citations and explanations 
see separate sheet 

VII. Certain defects in the international application 

The following defects in the form or contents of the international application have been noted: 
see separate sheet 

VIII. Certain observations on the international application 

The following observations on the clarity of the claims, description, and drawings or on the question whether the 
claims are fully supported by the description, are made: 
see separate sheet 
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V. Reasoned statement under Art. 35(2) PCT 

1 . Present independent apparatus claim 1 is not fulfilling the requirements under the 
PCT for lack of novelty (Art. 33(2) PCT). 

The examiner introduces Document US-A-4696041, denoted in this 
communication as D8. 

Document D8 (see in particular the abstract, I. 7-15) discloses (applying the 
terminology of present independent claim 1) an: 

"apparatus for predicting the speech level (see Title of D8) in an utterance of a 
speaker exposed to an environment containing a variable level of ambient 
acoustic noise (D8, col. 1 , I. 26-28), the apparatus comprising 

means for measuring said ambient acoustic noise level (D8, abstract, I. 7-10), and 

processing means for using said measured acoustic noise level to predict the 
likely speech level in said utterance (D8, abstract, I. 11-14)." 

This is exactly the wording of present claim 1 , which is therefore not novel and 
fails to fulfill the requirements of Art. 33(2) PCT. 

For the sake of completeness, it is mentioned that also International Search 
Report Document D6 ( EP-A-093303) implicitly takes away novelty of present 
independent claim 1. Indeed, D6 mentions the " means for measuring said 
ambient acoustic noise level" (D6, p. 4, 1. 15-16) and a gain controller in inverse 
proportion to the background noise controller (p. 4, I. 18-21). In the examiner's 
opinion this gain controller implicitly discloses that knowledge about the speech 
level to be expected is deduced from the measured noise level. For this reason 
present claim 1 is also not novel over D6. 

Nevertheless document D8 has been preferred, since it explicitly discloses the 
features of claim 1. 
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2. Present independent method claim 1 1 is corresponding to present independent 
apparatus claim 1 , and therefore fails to fulfill the requirements under the PCT for 
the same reason as explained above for claim 1 (lack of novelty, Art. 33(2) PCT). 

3. Present claims 2-5 are not novel over D6, and are hence not fulfilling the 
requirements of Art 33(2) PCT. 

The applicant is referred to D6, p. 14, I. 17 - p. 16, I. 12, as well as to Figs. 2 and 
3(A)-(G). See the indicated time period TO, being immediately adjacent to the 
utterance (claim 2), the "recognition switch sig" in Fig. 3(A) (claim 3), the time 
schedule in Fig. 3, particularly the buzzer signal (claims 4 and 5). 

Therefore all features of claims 2-5 are explicitly or implicitly (see above) 
disclosed in D6. Consequently, claims 2-5 are unacceptable for lack of novelty 
(Art. 33(2) PCT). 

4 Present dependent method claim 12 is corresponding to present dependent 

apparatus claim 3, and therefore equally fails to be novel over D6 (Art.33(2) PCT). 

5. No objections under the PCT are formulated against present claim 6, nor against 
present claim 1 3. 

As a matter of fact, none of the International Search Report Documents, nor 
Document D8, disclose or make obvious the use of previous speech level 
measurements while predicting the likely speech level of an utterance. 

The features of present apparatus claims 6 and 1 and, correspondingly, of present 
method claims 1 3 and 1 1 , should therefore have been combined to formulate 
new, allowable independent apparatus and method claims, respectively. 

6. No objections under the PCT are formulated against claim 7, nor would any 
objections under the PCT have been formulated against claims 8-9 either, 
provided that claim 8 had been corrected as indicated below. 
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7. Present independent apparatus claim 10 (see also the clarity objection in Sec. 12) 
and present independent method claim 14 fail to fulfill the requirements under the 
PCT for lack of novelty (Art. 33(2) PCT). As a matter of fact, these claims claim as 
only feature over claims 1 and 1 1 , respectively, the fact that the gain of a speech 
signal is controlled in accordance with the estimate of a speech level. This feature 
has already been shown above to be disclosed in D6 (see Section 2 of this 
communication). 

Consequently, present independent claims 10 and 14 are unacceptable for lack of 
novelty (Art. 33(2) PCT). 



VII. Certain defects in the international application 

8. The features of the claims are not provided with reference signs placed in 
parentheses (Rule 6.2(b) PCT). 

9. Contrary to the requirements of Rule 5.1 (a)(ii) PCT, the relevant background art 
disclosed in the documents D6 and D8 is not mentioned in the description, nor are 
these documents identified therein. 

It is noted that D6 is considered to be the closest prior art for the subject matter 
presently dealt with in claims 2-5, 10-12 and 14, whereas D8 is to be considered 
as the closest prior art for the subject matter presently dealt with in claims 6-9 and 
13. 

VIII. Certain observations on the international application 

10. The vague and imprecise statement in the description on page 6, 1. 3-4 ("extends 
to any inventive combination of the features set out above or in the following 
descriptions") implies that the subject-matter for which protection is sought may 
be different to that defined by the claims, thereby resulting in lack of clarity (Article 
6 PCT) when used to interpret them (see also the PCT Guidelines, lll-4.3a). 
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In order to overcome this objection, the above-cited wording should have been 
reformulated along the lines of "extends to the combination of the features as set 
out in the appended claims". 

1 1 . Present independent claim 1 0 fails to fulfill the requirements under the PCT, since 
this claim is not clear (Art. 6 PCT). 

As a matter of fact, this claim mentions "said level controlling means" on I. 12-13, 
although no level controlling means have been mentioned earlier in the claim. 

In order to be able to yet examine this claim, the examiner has assumed that "said 
level controlling means" would have been replaced by "said level". 

12. It is noted that present independent claim 10 actually contains all features of 
present independent claim 1 and should therefore have been formulated as a 
dependent claim referring to present independent claim 1 (R. 6.4.(a) PCT). See in 
this context the description, p. 5, I. 14-17. 

Similarly, present independent claim 14 actually contains all features of present 
independent claim 1 1 and should therefore have been formulated as a dependent 
claim referring to present independent claim 1 1 (R. 6.4. (a) PCT). See in this 
context the description, p. 5, I. 25 - p. 6, I. 1 . 

For reformulation of claims 10 and 14, however, see Sec. 7 of this communication, 
stating lack of novelty (Art. 33(2) PCT).. 

13. Taking into account that the function f(x) is intended to be e.g. a multiplying factor 
less than 1 (and, presumably, larger than 0), see the description, p. 8, 1. 13 and I. 
17-18, it appears that in the formula on p. 8, I. 9, "^No-N^" should actually have 
read "f^-No)". This error should have been corrected. 

Similarly, in present claim 8, "k(N 0 -N 1 ) ,, should actually have read "k^-No)". 

Furthermore, also in claim 8, "k>1" should apparently have read "k<1 H . 
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